Skip to content

Conversation

@rdhyee
Copy link
Contributor

@rdhyee rdhyee commented Nov 1, 2025

Problem

The "Color-code by type (sample/site/both)" button from PR #33 wasn't appearing on the live site.

Root Cause

Using //| echo: false on the viewof classifyDots cell was preventing the Observable runtime from properly rendering the button UI. The cell existed in the DOM but was empty:

<div id="ojs-cell-4" data-nodetype="declaration">
</div>

Solution

Changed from //| echo: false to //| echo: fenced, which:

  • Shows the code in a collapsible fence (like other code blocks)
  • Ensures the button UI renders properly in the Observable runtime
  • Maintains clean UX (code can be hidden by default if desired)

Changes

File: tutorials/parquet_cesium.qmd (lines 51-56)

Before:

//| echo: false
viewof classifyDots = Inputs.button("Color-code by type (sample/site/both)", {
  value: null,
  reduce: () => Date.now()
});

After:

//| echo: fenced
viewof classifyDots = Inputs.button("Color-code by type (sample/site/both)", {
  value: null,
  reduce: () => Date.now()
})

Testing

After deployment, verify at https://isamples.org/tutorials/parquet_cesium.html:

  • Button appears below the "Jump to Geocode" search box
  • Clicking button triggers classification query (~7 seconds)
  • Dots recolor: blue (sample locations), purple (site markers), orange (both)
  • Console shows: "Classification completed in XXXXms"

Impact

This restores the key performance optimization UX from PR #33:

  • Fast initial load with all blue dots (~2s)
  • Optional classification via button click (~7s)
  • User chooses whether to pay the classification cost

🤖 Generated with Claude Code

rdhyee and others added 5 commits October 31, 2025 13:37
Update both query paths to match Eric's authoritative query structure:

**Enhanced Queries**:
- Path 1 (get_samples_1): Now returns full sample metadata
- Path 2 (get_samples_2): Now returns full sample metadata
- Both now include: thumbnails, descriptions, alternate IDs, site info, coordinates
- Both use list_contains() for proper edge traversal
- Both order by has_thumbnail DESC (images first)

**Rich HTML Tables**:
- Replace raw JSON output with styled, scrollable tables
- Same 5-column layout as Eric's query: Thumbnail | Sample | Description | Site | Location
- Clickable thumbnail images or "No image" placeholders
- Clickable sample PIDs linking to OpenContext
- Clickable site names with "View site" links
- Formatted coordinates and descriptions
- Zebra-striped rows, sticky headers, result counts
- Loading and empty states

**Consistency**:
All three query result displays now use the same UI pattern:
- Eric's query (Path 1 only, authoritative)
- Path 1 query (direct event location)
- Path 2 query (via site location)

Users can now compare results across all three approaches with rich, visual data presentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Establish comprehensive testing foundation for future UI evolution:

**Test Framework**:
- Playwright for E2E browser testing
- Configured for Chromium (extensible to Firefox/Safari)
- HTML reporting with traces and screenshots
- CI-ready with automatic retries

**Test Coverage** (`tests/playwright/cesium-queries.spec.js`):
- ✅ Page loading and geocode search box
- ✅ Camera movement on geocode search
- ✅ HTML table structure (5 columns: Thumbnail | Sample | Description | Site | Location)
- ✅ Clickable sample PID links to OpenContext
- ✅ "View site" links
- ✅ Thumbnail images and "No image" placeholders
- ✅ Result count displays
- ✅ Empty state friendly messages
- ✅ Scrollable tables with sticky headers
- ✅ Zebra-striped rows
- ✅ Visual consistency across all three tables

**Test Data**:
- PKAP location with samples: `geoloc_04d6e816218b1a8798fa90b3d1d43bf4c043a57f`
- Larnaka site marker (empty state): `geoloc_7a05216d388682536f3e2abd8bd2ee3fb286e461`

**Infrastructure Files**:
- `playwright.config.js`: Test configuration with extended timeouts
- `package.json`: NPM scripts (test, test:headed, test:ui, test:debug)
- `tests/README.md`: Comprehensive testing guide
- `tests/playwright/cesium-queries.spec.js`: Full test suite

**NPM Scripts**:
```bash
npm test              # Run all tests
npm run test:headed   # Run with browser visible
npm run test:ui       # Interactive UI mode
npm run test:debug    # Debug mode with inspector
npm run test:report   # View HTML report
```

**Gitignore Updates**:
- node_modules/
- test-results/
- playwright-report/
- package-lock.json

**Future-Ready**:
This infrastructure provides a solid foundation for:
- Adding new UI feature tests
- Visual regression testing
- Accessibility testing
- Performance monitoring
- Cross-browser testing
- Mobile responsive tests

Tests validate the enhanced query UI (Path 1, Path 2, Eric's query) to ensure consistent, high-quality user experience as the UI evolves.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…nhancement

**Performance Improvement**: 7+ seconds → ~2 seconds (71% faster!)

## Problem
The expensive CTE query with JOIN + GROUP BY took 7+ seconds to classify
each geocode as sample_location/site_location/both before rendering any dots.
This made the page feel frozen and unresponsive.

## Solution: Progressive Enhancement
1. **Fast initial load** (~2s): Simple SELECT DISTINCT query, all dots blue
2. **Optional refinement** (~7s): Button to classify and color-code by type
3. **Chunked rendering**: 500 points per batch with progress indicator
4. **Performance telemetry**: Console logs show timing for all operations

## Technical Changes

### Simplified Initial Query (lines 137-163)
**Before** (expensive):
```sql
WITH geo_classification AS (
    SELECT geo.pid, geo.latitude, geo.longitude,
           MAX(CASE WHEN e.p = 'sample_location' THEN 1 ELSE 0 END) as is_sample_location,
           MAX(CASE WHEN e.p = 'site_location' THEN 1 ELSE 0 END) as is_site_location
    FROM nodes geo
    JOIN nodes e ON (geo.row_id = e.o[1])
    WHERE geo.otype = 'GeospatialCoordLocation'
    GROUP BY geo.pid, geo.latitude, geo.longitude
)
SELECT pid, latitude, longitude, CASE ... END as location_type
FROM geo_classification
```

**After** (fast):
```sql
SELECT DISTINCT pid, latitude, longitude
FROM nodes
WHERE otype = 'GeospatialCoordLocation'
```

### Optional Classification Button (lines 50-56, 769-845)
- User can click button to run classification query
- Updates existing point colors/sizes in-place
- Same color scheme as before:
  - Blue (small): sample_location_only - field collection points
  - Purple (large): site_location_only - administrative markers
  - Orange (medium): both - dual-purpose locations

### Chunked Rendering (lines 169-218)
- Render 500 points per batch
- Yield to browser between chunks (keeps UI responsive)
- Dynamic progress: "Rendering geocodes... 500/198,433 (0%)"

### Performance Telemetry (lines 376-384, 438-446, 500-508)
- Console logs for all queries: locations, Path 1, Path 2, Eric's query
- Example output:
  ```
  Query executed in 1847ms - retrieved 198433 locations
  Rendering completed in 423ms
  Total time (query + render): 2270ms
  ```

## User Experience Improvements

**Before**:
- Page frozen for 7+ seconds with static "Loading..." text
- No feedback on progress
- User uncertain if page crashed

**After**:
- Interactive in ~2 seconds with all dots visible
- Progress indicator shows "Rendering geocodes... X/Y (Z%)"
- Optional color-coding button if user wants classification
- Console telemetry for debugging and optimization planning

## Files Changed
- `tutorials/parquet_cesium.qmd` (+86 lines): Optimized queries + telemetry
- `OPTIMIZATION_SUMMARY.md` (new): Performance analysis and results
- `LAZY_LOADING_IMPLEMENTATION.md` (new): Technical implementation details
- `PERFORMANCE_OPTIMIZATION_PLAN.md` (new): Future optimization roadmap

## Testing
Test at http://localhost:XXXX/tutorials/parquet_cesium.html
- Expect: Page loads in ~2 seconds with all blue dots
- Click "Color-code by type" button → dots recolor after ~7 seconds
- Console shows timing for all queries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The button wasn't appearing on the page because echo: false was hiding both
the code and preventing proper Observable rendering. Using echo: fenced shows
the code in a collapsible section while ensuring the button UI renders.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@rdhyee rdhyee merged commit b39072e into isamplesorg:main Nov 1, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant